Linear Model Evaluation Based on Estimation of Model Bias

Model selection criteria often arise by constructing estimators of measures known as expected overall discrepancies. Such measures provide an evaluation of a candidate model by quantifying the disparity between the true model which generated the observed data and the candidate model. However, attention is seldom paid to the problem of accounting for discrepancy estimator variability, or to the companion problem of establishing discrepancy estimators with certain optimality properties. The expected overall Gauss (error sum of squares) discrepancy for a linear model can be decomposed into a term representing the estimation error, due to unknown model coefficients, and a term representing the approximation error, or bias, due to model misspecification. Since the first error term is seen to depend only on model dimension, a known quantity, the problem of estimating the expected overall Gauss discrepancy reduces to the problem of estimating a bias parameter. In this paper, we derive estimators of model bias with frequentist optimality properties and consider how confidence interval estimation can be used to quantify the uncertainty inherent to the problem of bias parameter estimation. We also show how the problem of estimating model bias can be approached from a Bayesian perspective. To illustrate our methodology, we present a modeling application based on data from a cardiac rehabilitation program at The University of Iowa Hospitals and Clinics.